\section{Results}
\label{results}

Since our primary goals were to research dependencies between different
characteristics of the bots and to research local maximums and niche
populations, we first decided to research how many bots in a population would
lead to the best evolution (as to prevent negative effects due to crowding) and
which fitness-function would give the best results. We decided to judge the bots
on five
characteristics \footnote{CHARACTERISTIC\_ATTACK\_SKILL,
CHARACTERISTIC\_REACTIONTIME, CHARACTERISTIC\_AIM\_ACCURACY,
CHARACTERISTIC\_AIM\_SKILL, CHARACTERISTIC\_ALERTNESS
(all of these should evolve
to 1.0 for the best bot, except reactiontime which should approach 0.0, as
clearly a lower reactiontime is better as it will mean the bot responds faster)
} which have shown to decide the skills of the standard bots mostly.

All of the experiments were run with the following settings:
\begin{itemize}
\item
Number of rounds: 200
\item
Roundtime: 10 min
\item
SwappointRate: 0.04
\item
CrossoverRate: 0.70
\item
MutationRate: 0.04
\item
Selection Model: Roulette selection
\end{itemize}
The only exceptions were some experiments we ran to investigate the effect of
longer roundtimes, which we ran using a roundtime of 40 minutes, and some
experiments we ran to investigate the influence of the completely random
mutation, for which we used a gaussion distribution was performed with a
standard deviation of 0.05 (with the mean being the previous value of the
characteristic that was being mutated)

\subsection{Different fitness-functions}

For our first experiment we tried to distinguish between the qualities of the
different fitness-functions. The different fitness functions use the following
parameters from the game, frags (the amount of kills the bot made), deaths (the
amount of times the bot died) and the ranking of the bot (the bot who has the
most frags will have the highest ranking).

The formulas for the four fitness functions used are the following:

\begin{itemize}
\item
Fitness 0: $ \frac{frags}{|frags|}(frags^{2}) - deaths^{2} $
\item
Fitness 1: $ \frac{frags}{|frags|}(frags^{2}) $
\item
Fitness 2: $ frags^{3} $
\item
Fitness 3: $ ranking^{2} $
\end{itemize}

\begin{figure}[!ht]
	\centering
	\subfigure[Evolution of attack skill using fitness function 0]
	{
		\includegraphics[width=.45\linewidth]{./images/characteristic_attack_skill-fitness0.png}
		\label{fitness0}
	}
	\subfigure[Evolution of attack skill using fitness function 1]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_attack_skill-fitness1.png}
        \label{fitness1}
    }
    \subfigure[Evolution of attack skill using fitness function 2]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_attack_skill-fitness2.png}
        \label{fitness2}
    }
    \subfigure[Evolution of attack skill using fitness function 3]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_attack_skill-fitness3.png}
        \label{fitness3}
    }
    \caption{Fitness functions results}
    \label{fitness}
\end{figure}

The plotted value is the average for the characteristic over the bots in the
round. We used 10 bots per round and all the experiments used the exact same
settings (except for the fitness function). We only show the plots of 1
characteristic here, but the other ones have similar structures. (see figure 
\ref{fitness})

\subsection{Interpretation of the results with different fitness-functions}

As can be seen in these pictures, no fitness function leads to (stable)
evolution. There are a lot of sudden steeps and falls. Although some
fitness-functions (2 \& 3) end with high values, it is clearly visible that they
fluctuate a lot earlier on, and that high values are not maintained during
the evolution. The steeps in the direction of the maximum (minumum for
reaction\_time) value are of course desirable, because they lead to the preferred
result. These can also be easily explained.

When an important characteristic mutates closer to the desired value, the
performance of the bot improves and thus will be assigned a higher fitness-score. 
This improves its chances of being selected for reproduction. In its turn this 
will lead to greater number of bots containing this value for the characteristic. 
This process will repeat itself until all or most bots have reached this value 
(the horizontal parts in the graph).

The decline of the values however is not wanted, and leads to an unstable
evolutionary process where improvements are countered by downgrades. This
process is also counter-intuitive taken the explanation of how improvements are
made. Downgrades in characteristic values are supposed to lead to a lesser
performance of the bots in-game, and thus a (much) lower chance to being
selected for reproduction. However, analysis of the experiments showed that bots
with significantly worse characteristic values than others were still able to
win the game with a big lead from time to time. This resulted in them being
assigned a higher fitness-score and thus be selected more for reproduction. This
led to a fast decline and was not restored in the following rounds.

It is of course also possible that even though one characteristic degrades,
other important characteristics improve, and thus result in a better bot
overall. This could explain the decline of one characteristic on itself.
Analysis however showed that this was not the case and that points of decline in
one characteristic were not countered by improvements in others.

\subsection{Different number of bots}

Because of the undesired results of the experiment with different
fitness-functions we decided to implement a gaussion distrubition mutation
operator, instead of a completely random mutation. The new mutation operator is
applied to all genes all the time, as opposed to the completely random
mutation which was only applied in 4\% of cases, and will thus prevent an exact
stabilization of values. It will also prevent values from changing too rapidly.

\begin{figure}[!ht]
    \centering
    \subfigure[Evolution of reactiontime with 4 bots]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_reactiontime-4b.png}
        \label{reactiontime4}
    }
    \subfigure[Evolution of reactiontime with 10 bots]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_reactiontime-10b.png}
        \label{reactiontime10}
    }
    \caption{Different number of bots}
    \label{reactiontime}
\end{figure}

The plotted value (figure \ref{reactiontime}) is the average for the 
characteristic over the bots in the round. We used fitness-function 0 because 
this seemed the most intuitive definition of fitness and is further used all 
the same settings (except the number of bots). We show only the plots of 1 
characteristic here, but again the other ones have similar structures. Even 
though the results of reactiontime with 10 bots here seem promising, the other 
characteristics did not evolve to their preferred value, or even got worse than 
the starting point.

\subsection{Interpretation of the results with different number of bots}

As can be seen with these experiments a stable improvement is still not
achieved. However, some important observations can be made here. The values of
the experiment with 4 bots fluctuate much more than those in the experiment with
10 bots. A smaller population clearly leads to a much more instable environment
and will prevent a population from improving or stabilizing. A bigger population
leads to a more stable environment. This is necessary for evolution and we
expect that a bigger population will lead to better results in the evolutionary
process. Because of the possibilities within the game we were limited to a
maximum of 10 bots, but most probably a scheme can be developed to use bigger
populations.

\subsection{Longer roundtime}

After the previous two experiments we decided to test effect of the roundtime
(standard 10 minutes) on the evolution. We expected that the phenomena where a
worse bot would beat better bots would be solved by prolonging the
round-time. A longer round-time would lead to more time for the better bots to
excercise their skills and possibly for a better spreading of the
fitness-function.

\begin{figure}[!pht]
    \centering
    \subfigure[Evolution of attack skill using roundtime 40]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_attack_skill-roundtime40.png}
        \label{roundtimeas}
    }
    \subfigure[Evolution of reactiontime using roundtime 40]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_reactiontime-roundtime40.png}
        \label{roundtimert}
    }
    \subfigure[Evolution of alertness using roundtime 40]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_alertness-roundtime40.png}
        \label{roundtimeal}
    }
    \subfigure[Evolution of aim skill using roundtime 40]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_aim_skill-roundtime40.png}
        \label{roundtimeai}
    }
    \subfigure[Evolution of aim accuracy using roundtime 40]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_aim_accuracy-roundtime40.png}
        \label{roundtimeaa}
    }
    \caption{Evolution using roundtime 40}
    \label{roundtime}
\end{figure}


\subsection{Interpretation of the results with longer roundtime}

The results of this experiment are quite diversive. We can see (figure 
\ref{roundtime}) a strong improvement in both the reaction time and the 
alertness, however both starting with a peek in the wrong direction. The aim 
skill and attack skill fluctuate around their starting point, and the aim 
accuracy even drops to the other opposite than its preferred value. Though we 
seem to see some improvement these results are far from conclusive. Intuitively 
we expect a better performance with longer round times. Further experiments 
should be done to prove this.


\subsection{Control Experiments}

To check if there were no deficits in our code and evolution with it was
possible, we decided to evolve the bots using the values of the characteristics
theirselves as criteria for the fitness function (see fitness function 4). This
was possible because we knew what the best values for the different
criteria were. For this experiment we used a number of 10 bots.

\begin{figure}[!pht]
    \centering
    \subfigure[Evolution of attack skill using fitness function 4]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_attack_skill-fitness4.png}
        \label{fitness4as}
    }
    \subfigure[Evolution of reactiontime using fitness function 4]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_reactiontime-fitness4.png}
        \label{fitness4rt}
    }
    \subfigure[Evolution of alertness using fitness function 4]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_alertness-fitness4.png}
        \label{fitness4al}
    }
    \subfigure[Evolution of aim skill using fitness function 4]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_aim_skill-fitness4.png}
        \label{fitness4ai}
    }
    \subfigure[Evolution of aim accuracy using fitness function 4]
    {
        \includegraphics[width=.45\linewidth]{./images/characteristic_aim_accuracy-fitness4.png}
        \label{fitness4aa}
    }
    \caption{Results from fitness function 4}
    \label{fitness4}
\end{figure}

As can be seen in these pictures (figure \ref{fitness4}) there is a clear 
evolution towards the preferred values. In about 30 rounds all characteristic 
values have neared their maximum within a range of 0.05. From that point on all 
values stay close to their preferred values but keep on fluctuating slightly 
because of the continous appliance of a gaussion mutation over these values. 
With this experiment we have proven that evolution is possible with our 
algorithms and even at a rapid rate. Given the results obtained with fitness 
functions based on the external behavior of the bots, it seems that the 
complexity of the game environment at our settings prevents the external 
behavior from being clearly representative of the ad hoc qualities of the bots. 
See the Future Work for recommendations about how to solve this.
